The present invention relates to a method and a system for capturing sequences of images with different magnification values.
A system for capturing images (still or moving) can be a module suitable for use in a camera, a videocamera, a camera phone or videocamera phone (mobile phones that can capture photos or video), an endoscope, a surveillance camera, a toy, a personal digital assistant, a computer or a computer tablet, a thermal camera, an ultrasound or MRI (magnetic resonance imaging) device, an X-ray machine, etc.
Such an image capturing system conventionally comprises a sensor having a plurality of light-sensitive elements called pixels, which convert a received amount of light into digital values, and a lens unit for focusing the light on the sensor.
The sensor can be, for example, a CCD (Charged Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), a CID (Charge Induced Device), an IRCCD (Infra-Red CCD), an ICCD (Intensified CCD), an EBCCD (Electron Bombarded CCD), an MIS (Metal Insulator Semiconductor), an APS (Active Pixel Sensor), a QWIP (Quantum Well Infrared Photodetector), an MQA (Multiple Quantum Well), a sensor sensitive to the light spectrum and/or to other electromagnetic spectral bands, or some other type of sensor. It may possibly be associated with a Bayer or panchromatic filter in order to obtain a color image.
Image capturing systems are known that comprise a lens unit, a digital image sensor, and a focusing module.
“Lens unit” is understood to mean a unit comprising one or more elements which may be lenses, mirrors, and/or diffractive elements. The lens unit can be of fixed (without zoom capability) or variable focal length, meaning it includes a mechanism for changing the distance between the sensor and the lens unit in order to zoom in on the scene to be captured.
A variable focal length lens unit is generally more complex, cumbersome, and expensive than fixed focal length lens units. There are several kinds of variable focal length lens units, some maintaining the focus when changing the focal length, and others not maintaining the focus when changing the focal length (varifocal). The focal length of a variable focal length lens unit is usually chosen by the user. For example, the user may choose a short focal length (for example 24 mm focal length for a 24*36 mm2 sensor) to capture a nearby monument, an average focal length for a portrait (for example a focal length of 100 mm for a 24*36 mm2 sensor), and a long focal length to capture a distant object (for example a focal length of 200 mm for a 24*36 mm2 sensor). The variation in focal length has the effect of causing a change in magnification of the digital image that is generally at the same proportions as the variation in focal length. The variation in focal length also has the effect of causing a distortion of the digital image that is distinct from the change in magnification. This distortion is not exactly the same throughout the digital image, due to the distortion from the lens unit which is a function of the focal length.
The focusing module allows varying the focus and is adapted for adjusting the system focus to obtain a sharp image of a portion of the image field. In the present description, the terms focusing and focus are used interchangeably. The focusing module can be manual or automatic, and in the latter case is called the autofocusing module. The focusing module is associated with a lens unit that is either variable focal length (with zoom capability) or fixed focal length (without zoom capability). A variation in focus corresponds to changing the blur spot, or in other words changing the size of the image of a point through the lens unit and on the sensor. Such a variation in focus can cause a variation in the magnification of the digital image that is generally much lower, although distinct and visible, than the variation in magnification produced by a change in the focal length of the lens unit. In the particular case of varifocal-type variable focal length lens units, the variation in focal length simultaneously produces a variation in focus and therefore, by varying the focal length, a first variation in magnification due to the variation in focal length and a second variation in magnification due to the variation in focus are achieved simultaneously. The variation in focus may result in a secondary local distortion of the digital image that is distinct from the variation in magnification. This distortion is not exactly the same throughout the digital image, due to the distortion of the lens unit which is a function of the focus.
The focusing can be managed by a “metering” system which may include a rangefinder, and/or by a phase-detection system which consists of diverting a portion of the luminous flux to two sensors and measuring the shift between the two images in order to determine the distance to the object. Such metering systems are cumbersome and expensive and are usually reserved for high-end devices.
In the absence of a metering system, the focus may be determined by successive iterations and relative measurements, running through different available configurations with the autofocusing module and measuring an image attribute such as contrast or sharpness in each of these configurations, for example for one of the color components in a color image. One example can be found in document FR-A-294 9003, which describes a system and method for capturing images with different modes of the autofocusing module.
Such focusing systems that use successive iterations are inexpensive and compact but have the disadvantage of slow focusing because they are iterative. In particular, when an image is out of focus, the autofocusing module does not know which way to adjust the focus of the lens unit before the first iteration.
When only capturing one image (a photograph of a scene), the image is typically captured after focusing. However, when capturing a sequence of images (video), a change in focus may occur between two successive images in a sequence. This produces a blurred image and a variation in magnification due to the variation in focus which is briefly visible on the screen of the device as well as in a video recording.
The invention relates more specifically to the capture of sequences of images of a scene with different magnifications from one image to another. “Magnification” is understood to mean the ratio between the size of the image of an object through the lens unit and the actual size of the object. A variation in magnification may be caused by changing the focal length of the lens unit when zooming in or out, or by changing the focus in a particular scene.
When changing the focus in a scene to be captured, a visual effect commonly known as lens breathing occurs, caused by the variation in magnification from one image to another in an image sequence. This visual effect can be particularly troublesome in certain applications, such as when capturing video image sequences for high definition applications and/or for 3D applications.
US-A-2009/0066832 describes a system for capturing a video image sequence, comprising a device which controls the speed of movement of the lens unit in a different manner depending on whether the image being captured is subject to magnification caused by electronic zoom. That patent proposes setting the autofocusing module to the magnification imposed by the zoom in order to lessen the visual effects of the zoom on the variations in sharpness. The system described in this document does not allow compensating for the visual effects of lens breathing.
WO-A-2008/134234 describes a system and method for capturing a video image sequence in which the images that are out of focus are rejected. Such a system requires a high capacity for capturing images, to allow discarding the blurred images with no significant loss of information.
US-A-2008/0187302 describes a system for capturing a video image sequence, comprising a second lens unit that applies an image correction in case of movement of a first lens unit. Such a system allows zooming in without affecting the focus of each image. Such a system with dual lens units is complex and expensive.
Some of the available “reflex” devices equipped with focusing systems based on phase detection, for example the Nikon D90®, disable the autofocusing module during image capture after focusing. Such a solution is not acceptable for capturing video image sequences because focus is no longer guaranteed if there is a change of scene.
More recently available “reflex” devices, such as the devices in the Sony® SLT series, keep the phase detection-based focusing system active when capturing a video sequence. However, the diverting of a portion of the luminous flux to the sensors of the focusing system decreases the light the image sensor is exposed to and may increase the noise in the captured images. In addition, these devices are subject to the lens breathing effects described above.
Embodiments of the invention eliminate some of the technical limitations described above. More specifically, the invention aims to compensate for the visual effects of lens breathing when capturing a sequence of images with variations in magnification, for example arising from variations in focus.
Embodiments of the invention provide rapid and continuous focusing for each image when capturing a sequence of images.